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O 

£N) ■ Abstract. For the relations generated by pair of differential operator expressions one of 

which depends on the spectral parameter in the Nevanlinna manner we construct analogs 
of the generalized resolvents which are integro-differential operators. The expansions in 
■ eigenfunctions of these relations are obtained 

<N 

Q_i ! Introduction 

We consider either on finite or infinite interval operator differential equation of arbitrary 
order 
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h[y]=m[f], t£l, l=(a,b)CR 1 (1) 
in the space of vector- functions with values in the separable Hilbert space H, where 



>■ l x [y] = l[y] - Xm[y] - n x [y], (2) 

oo ! 

qq, Z[y],m[y] are symmetric operator differential expression. The order of l X [y] is equal to r > 

CTV 0. For the expression m[y] the subintegral quadratic form m{y, y} of the Dirichlet integral 

m [y^y\ = Jx m {y^y}^ ^ s nonnegative for t El. The leading coefficient of the expression m[y] 
may lack the inverse from B(H) for any t E X and even it may vanish on some intervals. For 



the operator differential expression n\[y] the form n\{y, y} depends on A in the Nevanlinna 
manner for t EX. Therefore the order s > of m[y] is even and < r. 

In the Hilbert space L^(X) with metrics generated by the form m[y,y] for equation (HJ-© 
we construct analogs R(X) of the generalized resolvents which in general are non-injective and 
which possess the following representation: 

R(X) = / -^B- (3) 
Ju 1 V - x 

where is a generalized spectral family for which Eoo is less or equal to the identity operator. 
(Abstract operators which possess such representation were studied in |14j.) 
This construction is based on a special reduction of the equation 

l[y]=m[f] (4) 

to the first order system with weight. Here I and m are operator differential expressions which 
are not necessary symmetric (in contrast to ((2])). For construction of R(X) we also introduce 
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the characteristic operator of the equation 



, M= .mfi, te i. ( 5) 



In the case r = 1, n\[y] = H\(t)y (here the mentioned reduction is not needed) the resolvents 
R(X) was constructed in [20J. 

Further in the work we consider the boundary value problem obtained by adding to equation 
JT])-([2]) the dissipative boundary conditions depending on a spectral parameter. We prove that 
for some boundary conditions solutions of such problems are generated by the operators R(X) if, 
in contrast to the case s = 0, n\[y] = H\(t)y, the boundary conditions contain the derivatives 
of vector-function f(t) that are taken on the ends of the interval. 

In the work we calculate Ea, and derive an inequality of Bessel type. In the case when the 
expression n\[y] submits in a special way to the expression m[y] we obtain the transformation 
formulae and the Parseval equality. The general results obtained in the work are illustrated on 
the example of equation (TjTJ with coefficients which are periodic on the semi-axes. We remark 
that in the case n\[y) = it follows from [H| Q31 021 EH] * nat if X = M 1 , r > s and dimH < oo 
then En for equation (TJTJ with periodic coefficients on the axis have no jumps. (For r = s in 
the described case may have jump (see, e.g, [23J ) ) . We show that in contrast to the case 
Ti\[y] = if r = 1, dimH = 2 then E^ for equation ([I]) with periodic coefficients on the axis 
may have jump. 

In the case n\[y] = the results listed above are known [23], and R(X) is the generalized 
resolvent of the minimal relation generated by the pair of expressions l[y] and m[y]. For this 
case we show in the work that in the regular case all generalized resolvents are exhausted 
by the operators R(X), and thereby by virtue of |21| their full description with the help of 
boundary conditions is given. A review of other results for the case n\[y] = is in the work 
[22]. 

In the works [8] , [9] the question of the conditions for holomorphy and continuous reversibil- 
ity of the restrictions of maximal relations generated by l\[y] (|2|) with m[y] = 0, n\[y] = H\(t)y 
in L^ H ^ uyQXo (®^0 7^ 0) and also by the integral equation with the Nevanlinna matrix mea- 
sure was studied (using some of the results from [21]). We remark that the relations inverse 
to those ones considered in [8], [9] do not possess the representation ([3]). Also we note that 
the resolvent equation ([I])- ([2]) does not reduced to the equations considered in [8], [9]. 

Many question, that concern differential operators and relations in the space of vector- 
functions, are considered in the monographs [TJ [3j HJ US] ESI ESI ETJ [32] containing an extensive 
literature. The method of studying of these operators and relations based on use of the abstract 
Weyl function and its generalization (Weyl family) was proposed in |11[ [T2"| ITS"] . 

We denote by ( . ) and || • || the scalar product and the norm in various spaces with special 
indexes if it is necessary. 

Let an interval A C M 1 , / (t) (t e A) be a function with values in some Banach space B. 
The notation / (t) E C k (A, B) , k = 0, 1, ... (we omit the index k if k = 0) means, that in 
any point of A / (t) has continuous in the norm || • \\ B derivatives of order up to and including 
/ that are taken in the norm || • if A is either semi-open or closed interval then on its 
ends belonging to A the one-side continuous derivatives exist. The notation / (t) E Cq (A, B) 
means that / (t) E C k (A, B) and / (t) = in the neighbourhoods of the ends of A. 
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1. THE REDUCTION OF EQUATION (OQ) TO THE FIRST ORDER SYSTEM OF CANONICAL TYPE 

WITH WEIGHT. THE GREEN FORMULA 

We consider in the separable Hilbert space ~H equation (j4]), where I [y] and m [/] are dif- 
ferential expressions (that are not necessary symmetric) with sufficiently smooth coefficients 
from B (T~L) and of orders r > and s correspondingly. Here r > s > 0, s is even and these 
expressions are presented in the divergent form. Namely: 

k=0 

where l 2j = D^ Pj (t) & , hj-i = \D^ X {D Qj (t) + s, (t) D} D^\ Pj (t), qj (t) , Sj (t) G (Z, B {%)) , 
D = d/dt; m [f] is defined in a similar way with s instead of r and pj (t) , q~j (t) , Sj (t) G B (T~L) 
instead of pj (t) , qj (t) , Sj (t). 

In the case of even r = 2n > s, p" 1 G B (H) we denote 

Q(t,l)=(° iIn i ^=j,S(t,l) = Q(t,l), (7) 

H(t,l) = \\h a p\\lp =1 , h a p£B{U n ), (8) 

where I n is the identity operator in B {% n ) ; h\\ is a three diagonal operator matrix whose 
elements under the main diagonal are equal to \\q\-, • • • , ^q n -i), the elements over the main 
diagonal are equal to (— |si, . . . , — |s n _i), the elements on the main diagonal are equal to 
(— po, . . . , — p n -2, \s n Pn 1( ln — Pn-i) h\2 is an operator matrix with the identity operators I\ 
under the main diagonal, the elements on the main diagonal are equal to (0, 0, — ^Snp" 1 ), 
the rest elements are equal to zero; /121 is & n operator matrix with identity operators I\ over 
the main diagonal, the elements on the main diagonal are equal to (0, . . . , 0, I'P^Qn), the 
rest elements are equal to zero; /122 = diag (0, . . . , 0, p^ 1 )- 
Also in this case we denote 

W (t, I, m) = C*- 1 (t, I) { \\m a p |£ p=1 } C7" 1 (t, I) ,m a p G B (H n ) , (9) 

where m\\ is a tree diagonal operator matrix whose elements under the main diagonal are equal 
to (— |gi, . . . , — |<7„_i), the elements over the main diagonal are equal to • • • , |s„_i), 

the elements on the main diagonal are equal to (pQ, . . . , p n -l)j m i2 = diag (0, . . . , 0, |s n ), 
m 2 i = diag (0, . . . , 0, -|g n ), ™22 = diag(0, . . . , 0, p n ). 
Operator matrix C (t, I) is defined by the condition 

C (t, I) col {/ (t) ,/'(<),..., (*) , / (2n " 1} (<),..., / (n) (t)} = 

= col (t\l) , /W , . . . , (t\l) , (t\l),..., /N ( t |0} , (10) 

where (i|Z) are quasi-derivatives of vector-function / (i) that correspond to differential 
expression L. 



W (t,l,m) is given for the case s = 2n . If s < 2n one have set the corresponding elements of operator 
matrices m a p be equal to zero. In particular if s < 2n then 77112 = 77121 = 77122 = and therefore W (t, I, m) = 
diag (mn, 0) in view of (114[) . 
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The quasi-derivatives corresponding to I are equal (cf. |30| ) to 



,b1 



y"(t\l) = y®(t), j = o, ... 
y [n] (t 10 = 



PnV 



(n) 



5<Zr 



(n-1) 



"I, 

In 



(t|i) 



(j+i) 



2n + 1 



3=0, 



(11) 
(12) 

90 = 0. 
(13) 



At that l[y] = yM (i|Z). The quasi-derivatires (t\m) corresponding to m are defined in 
the same way with even s instead of r and pj , q~j , s j instead of pj , qj , Sj . 
It is easy to see that 



C(t,l) 



In 
C21 C22 



V C a /3 G B (H n ) , 



C2I) C22 are upper triangular operator matrices with diagonal elements (— \qi, 

, Pnj correspondingly. 
2n + 1 > s we denote 



\n-l 



and ((-1)" "Pn, 
In the case of odd r 



/ -. \71 — 2 

(-1) Pn, 



Q(t,l) 




(14) 

(15) 
(16) 



where B (T-L n ) 3 hu is a three-diagonal operator matrix whose elements under the main 
diagonal are equal to (§91, ^q n -ij, the elements over the main diagonal are equal to 
(— |si, . . . , — |s n _i), the elements on the main diagonal are equal to (—po, ■ —p n -i), the 
rest elements are equal to zero. B (7i n+1 , 3 hi2 is an operator matrix whose elements 
with numbers j, j — 1 are equal to I\, j = 2, . . . , n, the element with number n, n + 1 is equal 
to T}S n , the rest elements are equal to zero. B {% n , % n+1 ) 3 /121 is an operator matrix whose 
elements with numbers j— 1, j are equal to ii, j = 2, . . . , n, the element with number n+1, n is 
equal to \q n , the rest elements are equal to zero. B (% n+1 ) 3 /122 is an operator matrix whose 
last row is equal to (0, . . . , 0, —Hi, —p n ), last column is equal to col (0, . . . , 0, Hi, —p n ), the 
rest elements are equal to zero. 
Also in this case we denote 



W(t, I, m) = \\m a p\\ 2 at p =1 , 



(17) 



where mn is defined in the same way as mu (|9|). B lH n+ , T-L n j 3 m.12 is an operator matrix 



whose element with number n, n + 1 is equal to —-^Sn, the rest elements are equal to zero. 



B (T~L n , T-L n+1 ) 3 mil is an operator matrix whose element with number n + 1, n is equal to 
— \q n , the rest elements are equal to zero. B (% n+1 ) 3 77122 = diag (0, . . . , 0, p n ). 



(18) 



Obviously for H (t,l) ©, (US]) and PF(i,Z,m) ©, (02| one has 

if* (t, Z) = if (t, V) , (t, Z, m) = W (t, Z, m* 



See the previous footnote 
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Lemma 1.1. Let the order ofQl is even. Then 

%h (t, i) = w (t, i, -qi) = w (t, r, -m) . 



(19) 



Proof. Let us prove the first equality in (fT9|) for even r = 2n. Let us represent H (t, I) ((8|) in 
the form 



H(t,l)=A(t,l) + B(t,l), 



where 



B(t,l) = \\B jk \\l k=1 , B jk eB(U n ), 
Bn = diag (0, 0, s n p~ x q n l^) , B i2 = diag (0, 0, -is n p~ 1 /2) , 
B 2 i = diag (0, 0, ip^Qn/^) , B 22 = diag (0, 0, p~ l ) . 
In view of CO}, §Z\§ - ([23]) one has 

B (t, C (t, /) = ||%fe|| f fc=1 , u jk EB(H), 



(20) 

(21) 
(22) 
(23) 

(24) 



u n 2n = -is n /2, u 2n 2n = h, rest u jk = 0. 
Hence 



C* (t, I) B (i, I) C (t, I) = \\v jk \\f k=1 , v jk € B {%) , 

v n 2n = ~\ (s n ~ v 2n 2n = P* n , rest v jk = 0. 

Hence C* (t, I) (t, I) C (t, I) = C* (t, I) W (t, I, -9/) C (t, I) in view of ©, ©, (JTDJ), ® . 
The first equality in (|19p for even r is proved. Its proof for odd r follows from (fl~6|) . (fT7|) , 

One can see from the proof that 

W(t,l,$tl) = -$SH(t,l). (25) 
The second equality in (|19p is a corollary of (|25p and (p~8|) . Lemma [1,1 1 is proved □ 

For sufficiently smooth vector-function / (t) by corresponding capital letter we denote (if 
f(t) has a subscript then we add the same subscript to F) 



H r B F(t, l,m) 



( A/2 \ 

E ©/ (j) (t) © o < 

i=o / 

n-l \ 

E ©/ a) (t) ) © o . 

/(*), 

/ n-l 



0, r = 2n, r = 2n + 1 > 1, s < 2n, 

>09 (-i/ (n) W), r = 2n + 1 > l,s = 2n, 



(26) 



E ©/ {j) (<) 1 © ( E©/ [r ~ i] (*!*: 



r = l, 

r = s = 2n 



From now on in equation (Jl]) 

p- l (t)eB(H) {r = 2n);{q n+l {t)+s n+1 {t)y l eB{%) (r = 2n + 1) . 
Theorem 1.1. Equation (|4]) is equivalent to the following first order system 



- ( (Q (t) + 5 (i) y ' ) + H (i) y = W (t) F (t) 



(27) 
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with coefficients Q (t) = Q(t,l), S(t) = S(t,l) Q, fifty, H(t) = H(t,l) ©, (EJ, wci^i 
= VP(t, /*, m), and with F (t) = F (t, l*,m) that are obtained from ([17]) and (l26l) 
correspondingly with I* instead of I. Namely if y (t) a solution of equation ([4]) then 

y(t) = y{t, h m i f) = 

n—l \ / n \ 

E ®y U) (*))©( E © (y [r_i] (* 10 - / [s_il (* l m )) J » r = 2n 

n—l \ / n \ 

E ©y (j) (*) j © ( E © {y [r ~ j] (< 10 - (< H) 1 © Hy (n) (*)) > r = 2n + 1 > 1 

(here /W (t|m) = as fe < § ) 

y(t), r = l 

(28) 

is a solution of (|27p mitt i/ie coefficients, weight and F (t) menfioned above. Any solution 
of equation (f2T|) wit/i suc/i coefficients, weight and F (t) is equal to ([28]) , where y (t) is some 
solution of equation Q . 

Let us notice that different vector- functions f(t) can generate different RHSs of equation 
(f27|) but unique RHS of equation Q . 



Proof. We need the following three lemmas. 

Lemma 1.2. Lei L Q 6e a differential expression of I type and of order a. Let us add to L a the 
expressions of i k lk type, where k = a + 1, . . . , {$, with coefficiens equal to zero. We obtain 
the expressions Lp which formally has the order (3, but in fact Lp and L a coincide. Then for 
sufficiently smooth vector-function f (t) 



o, j = [^]+i,..., § 



(here f^(t\L\) is defined by hi Sty with r = 1). 

Proof. Proof of Lemma [1.21 follows from formulae (|12|) - (|13p for quasi-derivatives. □ 

Lemma 1.3. Let f (t) E C s ([a, f3] , TL) , y (t) is a solution of corresponding equation Then 
the sequence fk (i) £ C°° ([a, ft] , 7{) and solutions y^ (t) of equation (J4j) with f (t) = fk it) exist 
such that 

fk{t) — ► fit), y k {t) — > y(t). 

This is trivial consequence of Weierstrass theorem for vector-functions |33| and formula 
(1.21) from [ID]. 
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Lemma 1.4. Let vector-function f (t) G C s (Z, T~L). Then 
W(t, I*, m)F(t, l*,m) = 



( ^e'© (/ [s_jl (t\m) + {fl s -^ (t|m))') j 

©/I s / 2 ] (i|m)eo©...eo, 

^E®(f [s - j] (t\m) + (/[-i- 1 ] (t|m))') j 

©0©...+©0© (-i/N (t|m)), 
po (<)/(*)© © ... © 0, 

£ © (/ [s_jl (t|m) + (/ [s - J - 1] (t|m)) ) 



r = 2n + 1, r = 2n, < s < 2n 



r = 2n + 1, s = 2n > 
s = 



j=0 

©0' 



+ iT (t, I) (0 © . . . © © /M (i |m)) , r = s = 2?i 



(29) 



Let us notice that W(t, l*,m)F(t, l*,m) does not change if the null-components in F(t, I* , m) 
we change by any ^-valued vector-functions. 

Proof. Let us prove Lemma 11.41 for r = s = 2n. It is sufficient to verify that 
2 ,9=1) col {/ (t) , /' (t) , (i) , / (2n - 1} (t) , ... (t)} = 



C* (t,Z*) 



£ (/ [r - j1 (t \m) + (/['•-i- 1 ! (t |m))' H © . 



+ 



+H(t\l) (0©0ffi...ffi0ffi/ N (*|m))}. (30) 
But in view of @, (|12|) . (|13|) the left side of equality (|3U|) is equal to 



' n-l 



Y,®{f [r ~ j] (t\m)) + [f [r - j - 1] (t 



m 



Oe...ffiO®/ |n| (t\m). 



And hence equality (|30p is true since C (t, l*)[. . .] = [. . .] and the last column of C* (t, I*) H (t, I) 
is equal to col (0, 0, I\) in view of ©, (fT4"|) . 

The proof for r = 2n + 1 , s = 2n is carried out via direct calculation using (|17p , (|12p , (|13p . 

The proof for s < 2n follows from the case s = 2n consicered above, Lemmas 11.21 11.31 and 
fact that elements Ujk € B {%) of matrix W (t, l*,m) are equal to zero if s < 2n and i > s/2 
or j > s/2. Lemma [1.41 is proved. □ 

Let us return to the proof of Theorem 11.11 Let y (t) is a solution of equation . Then 

l -[(Q (t, I) y(t, I, m, 0))' + S (t, I) y' (t, I, m, 0)} - H (t, I) y(t, I, m, 0) = 

= diag(yW(t|0,0,...,0) (31) 

in view of formulae that are analogues to formulae (4.10), (4.11), (4.24), (4.25) from |24] , 
Using (|3ip and Lemma [1.41 we show via direct calculations that y(t,l,m,f) (|28p is a solution 
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of (127|) for r = s = 2n, r = In + 1, s = 2n. Therefore in view of Lemmas ll.2[ 11.31 y (t, I, m, /) 
is a solution of (127|) for s < 2n. 

Conversely let y (t) = col (yi, . . . , y r ) is a solution of (|27p . Let y (t) is a solution of Cauchy 
problem that is obtained by adding the initial condition y(0,l,m,f) = y (0) to equation 
(j4]). Then y (t) = y(t,l,m,f) in view of existence and uniqueness theorem. Theorem 11.11 is 
proved □ 

Let us notice that Theorem 11.11 remains valid if null-components of F(t,l*,m) we change 
by any "H-valued vector- functions. 

R 

For differential expression L = ^ i k L^, where L 2 j = Pj (t) , 

k=o 

L 2j -i = {DQj (t) + Sj (t) D} D*- x , we denote by 

L[f,g} = J^L{f,g}dt, (32) 

the bilinear form which corresponds to Dirichlet integral for this expression. Here 

[R/2] 

L{f,g}=^2 { p j (*) f U) (*) . 9® (<)) + 

j=0 

+ \ £ (*) / W (*) - (*)) - (Oi (*) (*) > S W) (*)) (33) 

Theorem 1.2 (On the relationships between bilinear forms). Let f (t) , y (t) , fk (t) , y^ (t) (k = 1, 

&e sufficiently smooth vector-function. Starting from these functions by the formulae \2b}) . \28\ ) 

we construct F(t, I, m), Fk(t,l,m), y(t,l,m, /), yk(t,l,m, fk). Then: 

1. 

(W(t, I, m)F 1 (t,l,m),F 2 (t,l,m))=m{f 1 ,f 2 }. (34) 
2. a) If the order of $sl is even, then 

(W (t, I, -SW) y(t, I, m, f) , y(t, I, m, /)) - 9? (W (t, l*,m*) y(t, I, m, /)) , F (t, l*,m) = 

= -m{y,y}-$(m*{yJ}). (35) 

b) 

m{yi,f 2 } -m{fi,y 2 } = 

= (W (t, l,m)yx (t, l,m,f 1 ),F 2 (t,l,m))-(W (t, l*,m) *i (t, l*,m), y 2 (t, l*,m*,f 2 )), (36) 



although for r = s the corresponding terms in the right- and left-hand side of (|35j) and (|36p do 
not coincide. 

Proof. 1. follows from pi. (fT7|> . (|2B|l . (|55|> . 

2. Let r = s = 2n. For convenience when using notations of (|26p type we omit the argument 
m. For example by F (t, I*) we denote F (t, l*,m). 
a) We denote 

JF(i, m) = co/ {o, 0, /l 2 ™" 1 ! (i |m) , / N (t |m)} € W. (37) 
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One has 

(W (i, I, -9Q y (i, I, m, /) ,y(t, I, m, /)) = (W (t, I, -SW) Y (t, I) , Y (t, I)) - 

- (W (t, I, Y (t, I) ,F(t,m))- (W (t, I, F (t, m) , Y (i, I)) + 
+ (W (t, I, -SM) F (t, m) , F (t, m)) = - (3Q [y, y] + 

+ 2K (pjT V n] (t|3Z) , /N ( t \m)) + 9 (p^/M (*M , (t\mj) . (38) 

Here the last equality follows from (I18p . (|34p . (|29p . (JSj) - On the other hand we have 

9 (W (i, l*,m*) y(t, I, m,f),F (t, I*)) = 9 (W (t, l*,m*) Y(t,l*),F (t, I*)) + 

+ Z(W (t, l*,m*) ((Y (t, l)-Y (t, I*)) - F (t, m)) , F (t, /*)) = 9 (m* {y, /}) + 

+ farV n] ( t |Q« Z ) ; /N ( t | m) A + 9 Cp-i/W (t j m) ; / [n] (t | m) , j _ (39) 



Here the last equality is proved similarly to (|38p taking into account that yl n ] (t\l) — y^ (t\l*) = 
2tyW (t|3tf). Comparing ((33J), O we obtain {35]). 

b) In view of J25J, ©, CLE]) and Lemma Owe have 

(W (t, /, m) yi (t, /, m, /i) , F 2 (t, I)) = m {yi (t, I, m, fi) , f 2 } - 

- (^F 1 (t,m),H(t,l*)col{o, ...,0,/| n] (i|m*)}) = 

= m{ yi ,/ 2 }- (p-Vf'^M./^CtK))- (40) 

Similarly 

(H/(M%™)iMM*),z7 2 (MW,/2)) = m {/i,y 2 }- (p^ 1 /^ 1 (* , /i" 1 (* ]fr** )) (41) 
Comparing (gDJ), (gl]) we obtain (|36|). 

For r = 2n + 1, s = 2n or r = 2n + 1 V 2n, s < 2n, the corresponding terms in ()35|) . (f36|) 
coincide in view of ([9]), (I17p . (I26p . (|28p . (|34p . For example in these cases 

(W (t, I, -%l)y(t, I, m, f) ,y(t, I, m, /)) = ((W (t, Z, -9/)) Y (i, Z) , Y (t, I)) = - (Off) {y, y} 

Theorem 11.21 is proved. □ 

Let us notice that Theorem 11.21 remains valid if null-components in F k (t,l,m), F(t,l* ,m), 
F\(t,l* ,m) we change by any ^-valued vector- functions. 

Theorem 1.3 (The Green formula). Let \ k , m^ (A; = 1,2) are differential expressions of I 
([6]), m type correspondingly. The orders of \ k are equal to r, the orders vn k are different in 
general, even and are equal to s k < r. Let y k (t) G C r ([a, /3] , %), fk (t) E C Sk ([a, f3] , T~L), 
and \ k [y k ] =m k [f k ], k = 1, 2. Then 

P 

dt 



rP fP rP 

/ mi{/i,y 2 }dt- / ml{yi,f 2 }dt- (h - 1%) {y u y 2 } 



1(Q (t,li) + Q* (t,l 2 ))yi(t, li, mi, /i), y 2 (t, 1 2 , m 2 ,/ 2 ) 



, (42) 

where Q(t,l k ), y k (t, l k , m k , f k ) correspond to equations l k [y) = m k [f] by formulae ((TJ), (|15D . 
(p8|) with l k ,m k ,y k , f k instead of I, m,y, f correspondingly. 

Proof. We need the following 
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Lemma 1.5. For sufficiently smooth vector-function g\ (t) , g 2 (t) one has 

((H (t, h)-H (t, l* 2 )) g x (t, l u mi, 0) , g 2 (t, 1 2 , m 2 , 0)) = 

= i~{h- l* 2 ) {91,92}, r = 2n 

{- Oi - II) {91,92} +(lL+l-C+i) {91,92}, r = 2n + l, 

where l^n+i are ^ e ana ^°9 s ofhn+i- 

Proof Let r = 2n. Then in view of fl2D])-([2ID, CED, (USD we have 

((# (t, li) - H (t, ID) g x (t, li, mi, 0) , ff 2 (t, 1 2 , m 2 , 0)) = 

= ((A (t, li) - A (t, T 2 )) 91 (t, li,mi, 0) , 92 (i, 1 2 , m 2 , 0)) + 

+ (C* (t,l 2 )B(t,l 1 )C(t,l 1 ) C oi{^,^ 1 , g[ nr - 1 \g? n - 1) , g[ n) } , 

COl ^2,92, 92 , 92 ,--,92 ])~ 

-(colfa,^...,^^,...,^}^* (tM)B(tM)C(t^)col{g 2 ,g 2 ,...,g^- 1) ,g { 2 2n - 1 \.. 

= -(li- 15) {/,</}• 

The proof of (|43p for r = 2n + \ follows directly from (fl"6|) . (|28|) . Lemma [1,51 is proved. □ 

Now Green formula (|42|) we obtain from the following Green formula for the equation (|27|) 
that correspond to equations lj. [y] = m^ [/] 

P 

(W (t, 1^, mi) Fi (t, lj, mi) , y 2 (t, 1 2 , m 2 , / 2 )) dt- 



(W (t, l* 2 ,m* 2 ) yi (t, li, mi, /i) , F 2 (t, 15, m 2 )) 
+ / ((fr(t,li)-ff(t,l5))gi(t,li,mi,/i),ife(t,l 2 ,m 2 / 2 ))cft- 

J a 

ft 3 i 

J - { ((5 (i, h) - Q* (t, l 2 ))y[ (t, h, mi, /i) , y 2 (t, 1 2 , m 2 , / 2 )) - 
- ((Q (i, h) - S* (t, 1 2 )) ft (i, li, mi, /i) , y 2 (t, 1 2 , m 2) / 2 )) } di = 

P 

■ (44) 



| (Q (t, h) + Q* (i, 1 2 )) m (i, li, mi, /i) , y 2 (*, 1 2 , m 3 , / 2 ; 



Let r = Sk = 2n. For convenience by F& (t, 1£) , lfc) we denote F^ (t, It, im,) , Yfc(i, 1&, m*.) 
correspondingly Then in view of ([8]), (|28p . (|29p . (|34p . (|43p one has: 



(W (f, 1^, mi) Fi (t, 1J) , y 2 (t, 1 2 , m 2 , / 2 )) = mi {/i, y 2 } + 

+ (FT(t,li) C oZ{0, ...,0,/i n] (t|mi)} jC oi{0, ...,0,y 2 n] (t 1 1 2 ) — t/^ 1 (t|lI)-/ 2 W 

(45) 

(i, 15, m|) m (t, h, mi, /i) , F 2 (i, 15)) = { yi , / 2 } + 

+ (H (t, 15) coZ {o, 0, yf 1 (t |li ) - y[ n] (t |l£ ) - /j n] (t |mi )} , col {o, 0, /f (i |m 2 )}) ; 

(46) 
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((H (t, h) - H (t, 1|)) yx (t, h, mi , h) , y 2 (t, 1 2 , m 2 , / 2 )) = - (h - I|) {</i, y 2 } - 

- ((H (t, h) - H (t, l* 2 )) Yy (t, h) , T 2 (t, m 2 )) - {{H (t, h) - H (t, Ji (i, mi ) , F 2 (t, 1 2 )) + 

+ ((if (t, li) - H (t, It)) col {0, . . . , 0, /J" 1 (t |mi)}, coZ {o, . . . , 0, /j" 1 (t |m 2 )}) . (47) 

where J- kit, m^) are the analogs of (|37p . 

Let us denote by Pj,Qj, Sj the coefficients of lj. Then in view of ([8]) 

(H (t, h)col {0, . . . , 0, f[ n] (* |mi )}, col {0, ... , 0, # (i |1 2 ) - # (t |1J )}) = 

= ((^)" 1 /J n] (t Inn ) , y [ 2 n] (t \l 2 ) - # (t |11 )) , (48) 

and 

(col {0, . . . , 0, yP it |li ) - yf 1 (t |1£ )} , H it, l 2 ) col {0, . . . , 0, f 2 [n] (i |m 2 )}) = 

= (VP it 111 ) - 2/[ nl (« |13 ) - iplr l f [ 2 ] it Im 2 )) • (49) 
On the another hand in view of ([8]), (|12p we have 

- HH it, h) - H it, l* 2 )) Y x it, h) , T 2 it, m 2 )) = 

= - (mrvjz - nj£r x %i2) y { r x) + OpI)- 1 - (^r 1 ) it \h ) , A n] it im 2 )) = 

= ((pIT 1 (y^ it lii ) - y [ i ] it |i 2 )) , ft ] it |m 2 )) , (so) 

where the last equality is a corollary of (fT2|) and its following modification: 

(Pj J/l \t\h)-y\ - -jyPn) QnVl 

Analogously it can be proved that 

i(H it, h)-H it, l* 2 )) T x it, mi) , Y 2 it, 1 2 )) = 

= (/W (t | mi ) , (p^)-i (# {t | l2 } _ y M {t |i* ))) . (51) 

Comparing (I44p - (|5ip we get (I42p since the last J*^ 3 in the left-hand-side of (|4"4"P is equal to 
zero if r = 2n in view of ([7]) . 

For s < r = 2n the proof of (02]) easy follows from (j2BJ, (J2BJ), (33]), (01]) in view of 

footnote [TJ 

Now let r = 2n+l. Then the last J" in the left-hand-side of (|44h is equal to (l 2n+1 — l 2 ^+i) {2/1)2/2} dt. 
Hence the proof of (@2J) for s < 2n <V = 2n+ 1 follows from ([17 ]) . (pJ|) . (|2"8"| . (|M ]1 . (pi) . (14%]) . 
Theorem 11.31 is proved. 

□ 

Remark 1.1. In view of Lemmas 11.21 H-31 all results of this item are valid if the condition of 
evenness of s is changed by the condition s < 2 [|] . 

2. Characteristic operator 
We consider an operator differential equation in separable Hilbert space Hi: 

% - ((Q(t)x(i))' + Q*(i)x'(i)) -H x (t)x(t) = W x (t)F(t), tel, (52) 



12 



VOLODYMYR KHRABUSTOVSKYI 



where Q (t) , [3ft Q (i)] -1 , H x (t) G B (Hi) , Q (t) G C 1 (X,B (Hi)); the operator function 
H\ (t) is continuous in t and is Nevanlinna's in A. Namely the following condition holds: 

(A) The set ^ 3 C \ R 1 exists, any its point have a neighbourhood independent of t G X, 
in this neighbourhood H x (t) is analytic W G X; VA G AH X (t) = iT|(i) eC(T,B (Hi)); the 
weight W x (t) = QH X (t) /QfA > (3fA / 0). 

In view of [21] V/i G A f| : ^ (*) = <9# A (i) /d\\ x=fl is Bochner locally integrable in the 
uniform operator topology. 

For convenience we suppose that G X and we denote 3ft Q (0) = G. 

Let X A (t) be the operator solution of homogeneous equation (l52j) satisfying the initial 
condition X\ (0) = /, where / is an identity operator in Hi- Since H\(t) = H^(t) then 

Xl(t)[*tQ(t)]X x (t) = G, A G A. (53) 

For any a, (3 G X, a < /3 we denote A A (a, /?) = X* (i) W A (t) X A (i) dt, 
A^ = {/i G T^i |/i G KerA A (a, /3) Va, /?} , P is the ortho-projection onto A r_L . iV is independent 

of A G A (ZD- 

For x (t) G Hi or x (t) e B (Hi) we denote C7 [x (t)] = ([3ftQ (t)] x (t) , x (t)) or J7 [x (*)] = 
x* (t) [3ft Q (t)] x (i) respectively. 

As in |20[ [2Tj we introduce the following 

Definition 2.1. An analytic operator-function M (A) = M* (A) G B (Hi) of non-real A is 
called a characteristic operator of equation ([52]) on X (or, simply, c.o.), if for 9 A 7^ and 
for any Hi - valued vector-function F (t) G (X) with compact support the corresponding 
solution x\ (t) of equation (f52~|) of the form 

x A (t, F) =TZ X F = jf X A (t) |m (A) - i S <7n (a - 4) (iG)' 1 ] X{ (s) W x (s) F (s) ds (54) 

satisfies the condition 

(3?A) lim (C/[x A (/3,F)]-C/[x A (a,X)]) <0 (3fA / 0) . (55) 

(a,/3)fX 

Let us note that in [21] c.o. was defined if Q(t) = Q*(t). Our case is equivalent to this one 
since equation (I52p coincides with equation of (l52l) type with 3ftQ(t) instead of C;(i) an d with 
H x (t) - ^Q'(t) instead of H x (t). 

The properties of c.o. and sufficient conditions of the c.o.'s existence are obtained in |20[l21j. 

In the case dim%i < 00, Q(t) = J = J* = J~ l , —00 < a = c the description of c.o.'s 
was obtained in |28j (the results of |28| were specified and supplemented in |22|). In the case 
dimT^i = 00 and X is finite the description of c.o.'s was obtained in [21 j . These descriptions 
are obtained under the condition that 

3A GA [a,/3]CX: A Ao (a,/3)>0. (56) 

Definition 2.2. [201 [21] Let M (A) be the c.o. of equation ([52]) on X. We say that the corre- 
sponding condition ([55|) is separated for nonreal A = fiQ if for any %i-valued vector function 
/ (t) G ,q (X) with compact support the following inequalities holds simultaneously for 

the solution x^ (t) (154j) of equation (I52[) : 

lim %n U [x M0 (a)] > 0, lim 9?// £/ [x^ ((3)} < 0. (57) 

ala 0Tb 
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Theorem 2.1. (20J [2Tj (see also Let M (A) be the cm. of equation f5l|) . We represent 
M (A) in the form 

M(A)= ^(A)-^(iG)- 1 . (58) 

T/ien i/ie condition \55}) corresponding to M (A) is separated for X = fiQ if and only if the 
operator V (^o) *s the projection, i.e. 

?W=P 2 W. (59) 

Definition 2.3. [201 [21] If th e operator-function M (A) of the form ([58]) is the c.o. of equation 
([52]) on X and, moreover, "P (A) = V 2 (A), then V (A) is called a characteristic projection (c.p) 
of equation ([52]) on X (or, simply, c.p). 

The properties of c.p.'s and sufficient conditions for their existence are obtained in |21| . 
Also |21J contains the description of c.p.'s and abstract an analogue of Theorem 12. 11 

The following statement gives necessary and sufficient conditions for existence of c.o., which 
corresponds to such separated boundary conditions that corresponding boundary condition in 
regular point is self-adjoint. This statement follows from Theorem 12.11 

Let us denote T~L + (%_) the invariant subspace of operator G, which corresponds to positive 
(negative) part of cr(G). 

Theorem 2.2. Let — oo < a. If P = I then for existence of c.o. M{\) of equation (|52p on 
(a, b) such that 

^oeC\K 1 : U[ XlM) (a, F)} = U[x,x a (a, F)} = (60) 

(and therefore condition (|55p is separated on A = fiQ, A = fio) it is necessary and sufficient 
that 

diniH + = diniH_ (61) 

(in (|60p x\(t,F) is a solution ()54h of (|52p which corresponds to c.o. M(A), L 2 w u\(a, b) B 

F = F(t) is any Tii-valued vector-function with compact support). If condition (|56p holds then 
condition (|60p is also sufficient for the existence of such c.o. 

Proof. Necessity. Since P = I we obtain 

U[X, (a)(I-V(i,o))] = U[X flo (a)(I-n^))]=0 (62) 

in view of the proof of n°2° of Theorem 1.1 from |21| . 

Let for definiteness Qfio > 0. Then in view of Theorem 2.4 and formula (1.69) from |21] , 
(pi) . (1621) and the fact that 

%\(X* x (a)$tQ{a)X x (a) - G) < 0, Aei (63) 

we conclude that X fl0 (a)(I — 'P(/io))'Hi and X^ (a)(I — V(fio))'Hi are correspondingly maximal 
K<5(a)-nonnegative and maximal KQ(a)-nonpositive subspaces which are 5?Q(a)-neutral and 
which are KQ(a)-orthogonal in view of Remark 3.2 from |21| . Theorem 12.11 and (|53p . Hence 

(X w (a)(/ - P(/i ))^i) [±1 = X m {a){I - VfaWx 

in view of [21 p. 73] (here by [_L] we denote 3ftQ(a)-orthogonal complement). Therefore X^ Q (a)(I— 
V{p{]))rii is hypermaximal K<5(a)-neutral subspace in view of [21 p. 43]. Thus we obtain that 
in view of [21 P-42] that diniH + (a) = dinrH_(a), where H±(a) are analogs of T~L± for ?R.Q(a). 
In view of (|63p X~^{a)1ij r {a) and X^(a)Ti-(a) are correspondingly maximal uniformly G- 
positive and maximal uniformly G-negative subspaces. Therefore Hi is equal to the direct 
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and G-orthogonal sum of these subspaces in view of (153p and p. 75]. Hence we obtain (IBTI) 
in view of the law of inertia (2J p. 54]. 

Sufficiency follows from Theorem 4.4. from |21| . Theorem is proved. □ 



It is obvious that in Theorem 12.21 the point a can be replaced by the point b if b < oo, but 
cannot be replaced by the point b if b = oo as the example of operator id/dt on the semi-axis 
shows. Also this example shows that condition (I60p is not necessary for the fulfilment of the 
condition U[x^ (a, F)] = only. 

In the case of self- adjoint boundary conditions the analogue of Theorem 12.21 for regular 
differential operators in space of vector- functions was proved in |29| (see also |31j). For finite 
canonical systems depending on spectral parameter in a linear manner such analogue was 
proved in |27| . These analogs were obtained in a different way comparing with Theorem 12.21 

Let us consider operator differential expression l\ of (|6|) type with coefficients pj = pj (t, A), 
qj = qj (t,X) , Sj = Sj (t, A) and of order r. Let —l\ depends on A in Nevanlinna manner. 
Namely, from now on the following condition holds: 

(B) The set 6 3 C \ I 1 exists, any its points have a neighbourhood independent on t G X, 
in this neighbourhood coefficients pj = pj (t, A) , qj = qj (t, A) , s = Sj (t, A) of the expression 
h are analytic Mt G X; VA G B, pj (t, A), qj (t, A), Sj (t, A) G C j (X, B {%)) and 

p~ x (t, A) G 5 (H) (r = 2n) , (q n+1 (t, A) + s n+1 (t, A))" 1 G B {%) (r = 2n + 1) , t G X; (64) 

these coefficients satisfy the following conditions 

Pj (t, A) = p* (t, A) , qj (t, A) = s* (t, A) , A G B (65) 

(([BSD <^ l x = l* H(t,l k ) = H(t,l%),\€B); 

in view of 11181 

V/io, • • • , /irr+ij G % : 

[r/2] . 



E (Pj (t,^)hj,hj) + ^ £ {( s j(*> A )^?A'-i) - (<Zj fy-iA')} 

j=0 j=l 



< 0, 



9A 

t G X, 9A / 0. (66) 

Therefore the order of expression 9/^ is even and therefore if r = 2n + 1 is odd, then 
(? m +i, s m +i are independent on A and = q^+i- 

Condition (|66p is equivalent to the condition: (3^a) {/j /} /3A < 0, t G X, 9A 7^ 0. 
Hence Wt,/ A ,-§^ = > 0, t G X, 3A / due to Lemma O and Theo- 



, where the coefficients 9pj ^^ ) 

— — , — ^ — of expression dl^/dfi are Bochner locally integrable in the uniform operator 
topology. 

Let us consider in %\ = 7~L r the equation 

\ ((Q (t, /a) y (*))' + Q* (t, /a) V' (tj) - H (t, l\) y(t) = W (t, h~^)F (t) . (67) 



sX J ~ SA 

rem 11.21 and therefore H(t,l\) satisfy condition (A) with A = B. Therefore V/z G B Pi M 1 
W(t, Ifj,, — Srfr) = 9H g\^ is Bochner locally integrable in uniform operator topology. Here 

in view of ©, ([IBJ G SfW ^ = = ^ 

dqj (t,fj) dsj (t,/x) 
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This equation is an equation of (|52l) type due to (I18p and Lemma \l. 11 Equation ([5]) is equivalent 
to equation (|67p with F (t) = F (t, l^, — §jr^J due to Theorem 11,11 

Definition 2.4. Every characteristic operator of equation (|67|) corresponding to the equation 
([5]) is said to be a characteristic operator of equation ([5]) on X (or simply c.o.). 

Let m be the same as in n°l differential expression of even order s < r with operator 
coefficients pj (t) = p* (t) , q~j (t) , sj (t) = q*- (t) that are independent on A. Let 

s/2 s/2 

V7i , . . . , ferr+ii G % : < ^ {pj (i) fy,-) + 9 ^ (*) foj-i, Tij) < 

i=o j=i 

/[r/2] . m 

3 I E (*> \)hj,hj) + \ Yu ((sj{t,\)hj,hj-i) - {qj{t,X)hj-x,hj)) 

t<El, 3fA ^ 0. (68) 



< 



Condition (j68|) is equivalent to the condition: < m {/, /} < {/) /} t £ I, 9A ^ 0. 
Hence 

< W (t, l x , m)<W (t, l x , -||) = h) tel, 9A/0 (69) 



due to Theorem 11.21 and Lemma 11.11 

In view of Theorem 11.11 equation ([1]) is equivalent to the equation 

\ ((Q {t, h) y(t))' + Q* (t, l x ) y (t)) - H (t, /a) y (*) = W (t, ^ ™) ^ (*, *a, m) , (70) 

where Q (t, l\) , # {t, l\) are defined by ©, ©, (US]), (US]) with / A instead of / and W (t, l~ x ,m) 
F {t, l\,rn) are defined by ([9]), (fT7|l (f26l) with instead of / and y{t)=y {t, l\,m, f) is defined 
by (I28p with l\ instead of I. 

In some cases we will suppose additionally that 
3A £ B; a, (3 G X, G [a, /3], the number 5 > 0: 



^Ar 



{y(t,A ),y(t,A )} dt > 6" \\Py (0, Z Ao , m, 0)|| 2 (71) 



for any solution y(t,\o) of ([5]) as A = Ao, / = 0. In view of Theorem 11.21 this condition is 
equivalent to the fact that for the equation (|67p with F (t) = 
3A G .4 = i3; a, /3 G X, G [a, /3] , the number 5 > 0: 

(A Ao (a,/3) 5 ,g) >S\\Pg\\ 2 , g e H r . (72) 

Therefore in view of [23j the fulfillment of (|7ip imply its fulfillment with 5 (A) > instead 
of 5 for all A G B. 

Lemma 2.1. Let M (A) &e a c.o. of equation for which condition (|71|) Zio/ds iuii/i P = I T . 7 
j/ X is infinite. Let 9A 7^ 0, 7-L r -valued F (i) G L^^ ^ m ^ (Z) (%i particular one can set 
F (t) = F (t, l x ,m), where f (t) G C s (X, , m [/, /] < 00 J. T/ien £/ie solution 

x x (t, F) = K X F = J X x (t) |m (A) - i S <7n (s - t) (iC)" 1 } X{ (s) W {s, l~ x ,m) F (s) ds 

(73) 
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of equation (I70p with F it) instead F (t,l x ,m) , satisfies the following inequality 

{X) <Z(K X F,F) L2 {x) 



TZ X F\\ 2 L2 m < Q (K X F, F) L2 (x) /9fA, 9A + 0, (74) 



'(t,i x ,m) 



where X x {t) is the operator solution of homogeneous equation (|70p suc/i i/iai (0) = I r , 
G = 1ZQ (0,l\); integral (|73p converges strongly if I is infinite. 

Proof. Let us denote 

K (t, s, A) = X x (t) |m (A) - l -sgn (s - t) (iG)" 1 } X\ (s) 

If (I7ip holds with P = I r if X is infinite, then in view of (I69p and |2H p. 166] there exists a 
locally bounded on s and on A constant k (s, A) such that 

V/iG?T: ||if(M,A)/i|| L 2 (x) < fc(s,A) ||/t|| (75) 

W(t,i x ,m) 

Hence integral (I73p converges strongly if I is in finite. 
Let F (t) have compact support and suppF (t) C [a, f3\. 
Then in view of ( 1421 1 



<ZfZ(W(t,l- x ,m)K x F,F)dt 



9A 

1 (KQ(t,l x )K x F,K x F) t 



< (76) 

2 9A 

where the last inequality is a corollary of n°2. Theorem 1.1 from |21( p. 162] and the following 
Lemma 2.2. Let F x is the set of "H T -valued function from ; _ m ^ (a, (3), 

I x (a,p)F= Xl(t)W(t,l- x ,m)F(t)dt, F (t) G F x (77) 

J a 

Then 

h(<*,/3)F G ^Ker J X\{t)W {t,l x ,m) X- X (t) dtl CN 1 . (78) 

Proof. Let h G Jver X? (t) W (i, ^, m) A^ (i) => W (t, m) AT X (i) /i = =^ I A (a, /?) 
The second enclosure in (f78|) is a corollary of condition (f69|) . Lemma 12.21 and inequality (|76|) 
are proved. □ 

Thus Lemma 12.11 is proved if I is finite. Let us prove it for infinite I. Let finite intervals 
(a n , f3 n ) f I, F n = XnF, where - is a characteristic function of (a n ,f3 n ). If (a, f3) C 
(a n ,/3 n ) then 



I^Aj 



in view of (f?5j) . Ij55]> . But local uniformly on i: (K x F n ) (t) -> (fc^F) (t), in view of jZ5 
Hence 



^ 2 / 



^Hl* , „ , (a>/3 ) < p^f ; • (79) 



w W 



a; 



3fA| 
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for any finite (a,/3). Hence (|79l) holds with X instead of (a,f3). In view of last fact lZ\F n — > 
1Z\F in L 2 , a; x (X). Hence ([Till is proved since it is proved for F n . Lemma 12.11 is 

proved. □ 



Let us notice that in view of |21| PM{X)P is a c.o. of equation ([5]), if M (A) is its c.o. 
Obviously the closures of operators 1Z\ corresponding to c.o.s M (A) and PM (A) P are equal 



111 B ( L W(t^,m) ^ ' L W (t,l x -^A^ ^ 



Let us notice what in view of (|68p l\ can be a represented in form ([2]) where 

I = m h n x =l\-l-\m; 9n A {/, /} /3?A > 0, t G X, 9A ^ 0. (80) 

From now on we suppose that l\ has a representation in ([2]), (|80p and therefore the order 
of n\ is even. 

3. Main results 

o 

We consider pre-Hilbert spaces H and H of vector-functions y (t) G Cq (X, %) and y (i) G 
C s (X, H) , m[y (t) , y (t)] < oo correspondingly with a scalar product 

(f(t),g(t)) m = m[f(t),g(t)}, 

where m [/, g] is defined by (|32|) with expression m from condition (|68|) instead of L. 
The null-elements of H is given by 

Proposition 3.1. Let f (t) G H. Then 

m [/,/] = & m [/] = /W (t) = ... = /^l (t) = 0, t EX. 

Proof. Let us denote by m(t) G -B (% n+1 ) the operator matrix corresponding to the quadratic 
form in left side of (f68|) . Since m (i) > one has 

m[f,f] = 0^m(t)col{f(t),...J^ 2 \0,...,o] = / W (t) = . . . = = 

□ 

Example 3.1. Let dim% = 1, s = 2, p\ (t) > 0, |gi (t)\ 2 = \p\ (t)po (t). Then for expression 
m the first inequality (J68]) holds and m {fo, fo} = for /o (i) = exp ^| Jq qi/pidtj ^ in view 
of Proposition 13.11 

By L^j (X) and (X) we denote the completions of spaces H and in the norm || • || = 

o ° 

•) m correspondingly. By P we denote the orthoprojection in (X) onto X^ (X). 

Theorem 3.1. Let M (A) is a c.o. of equation (|5]) 7 for which the condition (I71D wif/t P = I r 
holds ifX is infinite. Let $5 A ^ 0, f (t) G H and 

col{yj (t,X,f)} = 

X x (t) |m (A) - \sgn (s - t) (iC)" 1 } X{ (s) W (s, l~ x , m) F (s, l~ x , m) ds, Vj G U (81) 

be a solution of equation (|70p . that corresponds to equation ([1]), where X\ (t) is the operator 
solution of homogeneons equation (|70p such that X\ (0) = L r ; G = (0, l\) (ifX is infinite 



18 VOLODYMYR KHRABUSTOVSKYI 

integral (I8ip converges strongly). Then the first component of vector function (|81|) is a solution 
of equation ([T]). It defines densely defined in (I) integro- differential operator 

R(X)f = yi (t,XJ), f€H (82) 

which has the following properties after closing 
1° 

R* (A) = R (A) , 9A + (83) 

2° 

i2 (A) is holomorphic on C \ R 1 (84) 

5° 

9(i?(A)/,/) L2 m 

II* ( A ) /llUd) ^ mg) , ^ 0, / G 1& (X) (85) 

Let us notice that the definition of the operator R (A) is correct. Indeed if f (t) G iif, 
m [f, f) = 0, then R (A) / = since W (i, / A , m) F (t, £ A , m) = due to ([MD, (1691). 

Proof. In view of Lemma[2J]integral (|8ip converges strongly if X is infinite. In view of Theorem 
ll.ll ^i (i, A, /) (|82p is a solution of equation JT]). 
In view of (15511 . (15511 



I* ( A ) J Wl? m {a,p) ^ < II* (A) / Ilia Ma („,/?) ^7 



||^ A F(t,^,m) 



a/ 



3= (JZ\F (t, l- x , m) , F(t, , l- x ,m) L 2 , p) 



In view of Lemma f2. II a nonnegative limit of the right-hand-side of ([86]) exists, when (a, (3) f X. 
Hence (J85]) is proved. 

Let 7T-valued F(t) G X^ (t/ _ m) (X). Then in view of jg9j, LemmaEU ^TgJ) one has 



ll^ll^ m < ||tt A F|f 2 < " v "*""' , (87) 

H^A F lli 2 (z) < ||^ A F|| L 2 (z) = ||^ A F|| L 2 (x) . 

In view of (j8"Tl) , f|88|) we have 



ir 



(t,; x ,m) 



II^aX|| L 2 .x)<||X|| L 2 ,(z)/l»A|, (89) 

II^aF|| L 2 (i) < ||F|| i2 (z) / |9fA|. (90) 

Let F (t) G L/^^ tl _ m ^(T), G (t) G L^ t ^ m ^(Z) are % r -valued functions with compact 
support. We have 

(^aX,G) L 2 {i) = (F,K- x ,G) l2 {I y (91) 
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since M (A) = M* (A). Due to inequalities ([92]) . ([93]) equality ([88]) is valid for F (t) , G(t) 
with non-compact support. 

Now it follows from, ([36]), ([94]) that V/ (t),g(t) G F 

m [-R (A) f,g]-m [f, R (A) <?] = (1Z X F (t, l x ,m) , G (t, l x ,m)) L 2 (I) - 

- (F (t, l- x ,m) , K X G (t, l x ,m)) L2 {I) = 

Thus the closure of the operator R (A) / in L^ (I) possesses property (1831) . 
Since in view of (|85p for any f (t) ,g (t) G H 

(R W fi 5 , )l2 1 (o,/3) ~^ (R W /; d)Lf n (X) ^ (a^)tl 

uniformly in A from any compact set from C \ R , we see that, in view of the analyticity of 
the operator function M (A) and vector-function W (t,l x ,m) F (t,l x ) (see (|29f) with I = l x ) 
the operator R (A) depends analytically on the non-real A in view of |16[ p. 195]. Theorem 
13. H is proved. □ 

For r = 1, n x [y] = H x {t)y Theorem 13.11 is known [20] . 

° o 

Let us notice that if = L^(I) then Theorem 13.11 is valid with f{t) G H instead of 

f{t) G H and without condition (|7ip with P = I r for infinite X. 

The following theorem establishes a relationship between the resolvents i?(A) that are given 
by Theorem [34] and the boundary value problems for equation ([TJ, ([2]) with boundary condi- 
tions depending on the spectral parameter. Similarly to the case n x [y] = |23] we see that the 
pair {y, /} satisfies the boundary conditions that contain both y derivatives and / derivatives 
of corresponding orders at the ends of the interval. 

Theorem 3.2. Let the interval I = (a,b) be finite and condition (I7ip with P = I r holds. 
Let the operator- functions A4\,J\f\ G B {ri r ) depend analytically on the non-real X, 

M\[VlQ(a,l x )]M x =N x *[VtQ(bJ x )]N x (SA^O), (92) 

where Q (t,l x ) is the coefficient of equation (|70p corresponding by Theorem \l.l\ to equation ([T]), 

\\M x h\\ + \\M x h\\ > (0 ^ h G H r , 9A ^ 0) , (93) 

the lineal {Ai x h ® M\h \h G rU } C H is a maximal Q-nonnegative subspace if QX 7^ 7 
where Q = (9A) diag (KQ (a, ^a) , — (fr, ^a)) (and therefore 

%\(N* x [KQ(b,h)}N x -M* x ['RQ(a,l x )}M x )<0 (3A ^ 0)) . (94) 

TTien 

1° . For any f (t) € H the boundary problem that is obtained by adding the boundary condi- 
tions 

3h = h (A, /) G H r : y (a, A, m, f) = M x h, y(b, A, m, /) = M x h (95) 

to i/ie equation ([T]) ; where y(t,X,f) is defined by (I28p . /ias i/ie unique solution R(X) f in 
C r (I,rl) as 9A 7^ 0. is generated by the resolvent R(X) that is constructed, as in Theorem 
\3.1\ using the c.o. 

M (A) = -i (X- 1 (a)M A + XT; 1 (6)7Va) (X' 1 (a)M x - X' 1 (b)^)" 1 (iG)' 1 , (96) 
where 

(X- 1 (a)M x -X x 1 (b)M x y 1 eB(H r ) (3A^0), 
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X\(t) is an operator solution of the homogeneous equation (I70p such that X\ (0) = I r . 

For any operator R(X) from Theorem \3. 1\ vector-function R{X)f (f G H) is a solution 
of some boundary problem as in 1° . 

Let us notice that if f (t) = g (t) then in boundary conditions (I95| : y(t,l,m,f) = y(t,l,m,g) 
in view of (I28D and Proposition 13.11 



Proof. The proof of Theorem l3.2l follows from Theorems 1 1 . 1 [ [37X1 and from |21[ Remark 1.1]. □ 



For the case n\ [y] = 0, Theorem 13.21 is known |21] , 

The example below show that the following is possible: for some resolvent R (A) from 
H 

Theorem 13.11 3fn (t) ^ such that m [fo] = and therefore the "resolvent" equation ([1]) for 

H 

R (A) fo is homogeneous but R (A) fo 7^ 0, SA 7^ 0. 

H 

Example 3.2. Let m in ([I]) be such expression that equation m [/] = has a solution fo (t) 7^ 0. 
Let in Theorem 13.21 A^a = ( ^ ^ ' "^" A = ( 1 ' * s corres P on ding resol- 

vent. Then i?(A)/ 7^ 0,9?A + 0, while if 7W A = ( ® ^ J , A/a = ^ J ) then f ° r the 

corresponding resolvent R (A) /o = 0, 3A 7^ (and therefore in view of |14[ p. 87] E^fo = 
for generalized spectral family En, which corresponds to R(X) by Q). 

It is known [T3] p. 86] that the operator-function R(X) ([83]) - ([85]) can be represented in the 
form 



^(A) = (T(A)-A)-\ (97) 
where T (A) is such linear relation that 

%T (A) < (max) , T (A) = T* (A) , AG C+ 



the Cayley transform (T (A)) defines a holomorphic function in A G C+ for some (and hence 
for all) /i G C+. Applications of abstract relations of T(A) type (Nevanlinna families) to the 
theories of boundary relations and of generalized resolvents are proposed in |12[ I13| 



The description of T (A) corresponding to R (A) from Theorem 13.11 in regular case gives 
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Corollary 3.1. Let X is finite and condition (171|) with P = I r holds. Let us consider the 



relation T (A) = T' (A) as QA 7^ 0, where 



' (A) = \ {y (t) J(t)}\y (t) L ^ X) y (t) e <T (X) , / (t) ^ / (t) G , (Z - n A ) [y] = m [/] 



y (i, Z — n\,m, f) satisfy boundary condition 
3h = h (A, /) eH r : y(a,l - n x ,m, f) = M\h, y(b,l- n\, m, f) = N\h, 
where operators M\,J\f\ satisfy the conditions of Theorem \3.fy 

def 

y(t,l -n x ,m,f) = y(t,l\,m, f)\m=0inl x = 

E ®y (j) (*) © E©(y [s '" jl (*K-n A )-/ [s ^ (t|m)) , 

;=o / .7=1 v 7 



'n-l 



2n 



n-l 



E ©y (i) (*) 



' (^E © {V [r ~ j] (t\l-n x )- f [s ~ j] {t\m))j (-iyW (t)) , r = 2?i + 1 > 1 

(/iere s' = order of expression I — n\, 
y [Q](t\l - nA ) = _| ( ft ( tj A) - A)) y m s' = 1 

y[ fe 'l (f |Z -ra A ) = os A/ < l^l , /W (t |m) = as k < f) 



i°. (T (A) — A)" 1 is eguaZ to resolvent R(X) (|8ip . (|82p /rom Theorem \3 . 1\ corresponding to 
c.o. M (A) (J96|). 

5°. Let R(X) is resolvent flST}, ((82]) /rom Theorem\3Jl Then R(\) = (T (A) - A)" 1 , w/iere 
T (A) is some relation as in item 1° . 

Proof. The proof follows from (128p . Lemma [1.21 Theorem 13.21 and Remark 1.1 from |21| . □ 

Let in (pQ), ([2]) n A [y] = I.e. l\ = I — Am, where Z = I* , m = m* and coefficients of 
expressions m satisfy condition (1551) . 

We consider in (X) the linear relation 

4 = j {y (t) , y (*)} |y (*) ^ y (t) ,5 (t) Ll ^ X) g(t),y (t) e C r (X, H) ,g(t)eH,l [y] =m[g], 

y (t, I, m, g) is equal to zero in the edge of I if this edge is finite and y (t, I, m, g) 
is equal to zero in the some neighbourhood of the edge of X if this edge is infinite 1 (99) 

where y (t, I, m, g) is defined by (I98p with l\ = I — Am, / = g. 

Below we assume that relation C consists of the pairs of {y,g} type. 

The relation C' Q is symmetric due to the following Green formula with A& = 0: 



Let us notice that vector-function g (i) in (I99|) may be non-equal to zero in the finite edge or in the some 
neighbourhood of infinite edge of X. 
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Let y k (t) G C r ([a, /3] , U), f k (t) G C s ([a, f3] X k G C, I [y k ] - X k m [y k ] = m[f k ], k 
1,2. Then 



r/3 r/3 _ r(S 

/ rn{fi,y 2 }dt- m {yi, f 2 } dt + (Ai - A 2 ) / m{yi,y 2 }dt 

J a J a J a 



= i (KQ (i, / A ) yi (t, *Ai > m, /i) , m (t, l X2 ,m, f 2 ))f a , (100) 

where y k (t, l\ , m, /^) for A^ G M 1 is defined by (|98p with l\ = l — Am. 

This formula is a corollary of Theorem 11.31 if QX k ^ 0. For its proof for example in the case 
Ai G M 1 we need to modify (142 p for equation I [yi] — (Ai + ie) m[y] = m [f± — iey±] and then 
to pass to limit in (I42D as e — > +0. 

In general the relation C' Q is not closed. We denote Cq = C' . 

Theorem 3.3. Let l\ = I — Am and the conditions of Theorem \3.1\ hold. Then the operator 
R(X) from Theorem \ 3.1\ is the generalized resolvent of the relation Cq. Let T be finite and 
additionally the condition ([7T]) hold. Then every such generalized resolvent can be constructed 
as the operator R (A) . 

Proof. In view of |14] and taking into account properties (I83p -( j85l) of the operator R{X) it is 
sufficiently to prove that R (A) (Cq — A) CI, where I is a graph of the identical operator in 
(X). But this proposition is proved similarly to |21} p. 453] taking into account (llOOp and 

the fact that in view of (jlOip (y — y)(t, I — Am, m, 0) = lj(t, I — Am, m,g — Xy) — y(t, I, m, g) 
if {y,g}eC' ,y = R(X)(g- Xy). 

Conversely let I be finite, R\ a generalized resolvent of relation Cq. We denote N\ = 
{y (t) G C r (X, H) , A G B, I [y] - Am [y] =0}. We need the following 

Lemma 3.1. Let condition (iTTI) hold. Then the lineal N\ is closed in L^ (X). 

Proof. The proof of Lemma 13.11 follows from (|34|) . □ 

Lemma 3.2. Let XeB. Then R (C - A) = N^. 

Proof. Let x (t) G N\, f (t) G H, y (t) is a solution of the following Cauchy problem: 

l[y}-Xm[y} = m[f], y(a,l x ,m,f) = 0. (101) 

Then 

m[f,x]=i(^Q(b,l x )y(b,l- x ,m,f),x(b,l x ,m,0)) (102) 



m view of Green formula ([TOO]) . Therefore R (C - X) C N x . 

Let g (t) G N£. Then 3 H 3 g n L W ] g, g n = x n @ f n , x n G N x , f n G iVf f n G H. Let 
y n be a solution of problem (|10ip with f n instead of /. In view of (j!02p with / = f n , one has: 

LUX) 



y n (b, l x ,m,f n ) = => f n G R (C - A) . But f n ^ g. Therefore R (£' - A) 5 iVf . Lemma 
is proved. □ 



Lemma 3.3. Let the condition flTJJ hold, X G B. Let [y,f \ G C* - X, f L "^ X) f G H. Then 
L 2 (X) 

y '= y £ C r (X, %) and y (t) satisfies the equation ([JJ . 

Proof. Let C r (T,TL) 3 yo be a solution of (JJ. Let G £q — A. Then (a,l x ,m,ip) = 

(p (b,l x ,m,if)) = in view of (|98p . (f99|) . Hence m[ip,f] = m[ip,yQ] due to Green formula 
(|100p . But m /] = (V')2/)z / 2 (j) in view of the definition of the adjoint relation. Hence 
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{ip, y — yo) = 0. Therefore y — yo "= y — yo G N\ in view of Lemmas 13.11 13.21 Hence 
y "= y G C r {X,T~C) and y is a solution of dTJ. Lemma 13.31 is proved. □ 



We return to the proof of Theorem 13.31 

Let / G 77. Then in view of Lemma 13.31 R\ f *== ^ y G C r (2^,77.) and y satisfies equation 
([1]). Therefore taking into account Theorem 11.11 |10[ p. 148] and (j53[) we have 

y (t) = [X x (t)^ L-^ (iGy 1 (J* sqn{s - t)X{ (s) W (s, l h m) F (s, Z A , m) ds^j } , (103) 

where [X\ (t)] 1 G B (77 r ,77) is the first row of operator solution X\ (t) from Theorem 13. 11 that 
is written in the matrix form, h = h\ (/) G A r_L is defined in the unique way in view of AMI) 
and condition (|7ip . 

Let us prove that h depends on I\f d = f^X^ (s) W (s, Z A , m) F (s, Z A , m) ds in unique way. 
Operator (7 A : 77 ->■ N L ) in view of Lemma O Moreover I X N^ = N L i.e. V/i G iV- L 3/ G 
77 : ho = I\fo- For example we can set 

/o = /o(<, A) = [X X (t)] x {A A (1) \ N± y 1 h (104) 

and to utilize the equality. 

W (s, l- x ,m) F (s, l- x ,m) = W (s, l x ,m) X~ x (s) {. . .y 1 h 

If / (t) ,g (t) G 77 are such functions that I\f = I\g, then in view of (j!03|) 



(pRQ (t, Ay (t, l x ,m, f - g) , Ay (t, l x , m, f - g) ^ 

= 3A ((»Q (t, Z A )) X A (t) (/i A (/) - h x (g)) , X A (t) (h x (/) - /i A , (105) 

where Ay = R\ [f — g] . But in view of (|100p the left hand side of f|105j) is nonpositive since 
R\ has property of (|85p type. The right hand of f|105j) is nonnegative in view of (j42]). Hence 
h\ (/) = h\ (g) in view of (|42p . ([7T]) . Thus /i depends on 7 A / in unique way and obviously in 
the linear way. Therefore 

h = M(X)I x f, (106) 

where M (A) : A r_L — > N 1 - is a linear operator and so R\f (/ G 77) can be represented in the 
form (1821) . 

Further, for definiteness, we will consider the most complicated case r = s = In. 
Let us prove that M (A) G B (N^) , 9A / 0. Let /i G , y = R x f , where / = f (t, A) 
see (1104p . Then in view of (11031) and Theorem 11.11 we have 

X x (t) M (A) /i = y (t, /a, m) - Jb (t, m) - \x x (t) (iGy 1 (J A (a, t) - 7 A (i, 6)) F , (107) 

where y (i, l\,m) , To = 7b (t, l\,Tn) , J-"o (i, m) are defined by (f26|) . ([37]) correspondingly with 
y and /o correspondingly instead of /, 7 A (0, t) Fq is defined by (|77|) . Therefore 

A A (a, 6) M(A)/t = 7 A y - 7 A (a, 6) LF (t, m) + ^X x (t) (iGy 1 (7 A (a, i) - 7 A (t, b)) F 

(1081 
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where F x y, I x (a, b) (...) G N 1 in view of (JTSJ), But 

\/g£H r : \(I- x y,g)\<m^\\X x (t)\\y \\W (i, l x , m)\\ dtX ' WRxhWi^U 



in view of Cauchy inequality and (|34J) . Therefore 

3 constant c(A) : |(I A y,<?)| < c(A) \\y\\ \\g\\ (109) 

since 

\\RxM\lUt) < II A a (o. &)ll 1/2 |(As (a, 6) Iat-l IIM / |3A| 

in view of (|34l) . (|109p and inequality: ||-Ra/o|Il 2 (X) — II/oIIl 2 (X) / 1^1- 

Obviously \(I X (a, b) (. . .) ,g)\ satisfies the estimate of type (fT09j) . Therefore M (A) £ ^(iV- 1 ). 
Now we have to prove that M(A) is a c.o. o equation (I67D . 

Let us prove that M (A) is strogly continuous for nonreal A. To prove this fact it is enough 
to verify it for A A (a, b) M (A); while the last one obviously follows from strongly continuity of 
vector-function I x R\fo(t, A). 

In view of (13411 we have Vo € % r 



(I x Rxh (t, A) - J^/o (t, fi),g) = m [R x f (t, A) , [X x (i)] x <?] - ?n [i* M / (t, M ) , [X„ (% g] . 
Then the required statement can be derived from the equality 

m { [X x (t) - X, (t)], g, [X x (t) - X^ (t)] l9 } = 
= (W (t, l x ,m) ((X x (t) - X^ (t)) g + (A - /i) JF(t, m)) , {X x (t) - X^ (t)) g + (A - M ) m)) , 

where m) is defined by fl37]) with / (i) = [X„ (i)] , 5, ||X A (i) - (i)|| -> uniformly 

ft— >A 

in t E [a, 6]. and from the analogous equality for m{fo(t, A) — fo(t, fi), fo(t, X) — /o(i, /•*)}• 

Let us prove that M (A) is analytic for nonreal A. To prove this fact it is enough in 
view of strongly continuity of M(A) to prove the analyticity in A of (Ix^M (A) I\f,g), where 
/ (t) e C r (X, H) , g e W, (3A)(3/x) > 0, 

/ v = ^ X; (t) (t, ^, m) X A (t) dt E B (AT- 1 ) , 

J7 1 £ 5 (X- 1 ) if I A - /x| is sufficiently small. In view of (fTUHt (182]) . Theorem O (pij) . (|29|) . 
([8]) we have 

(J^M (A) l x f, g) = m [R x f, [X„ (t)] t g] + (A - /x) ^ ((i? A /) N (< |m) , ^ (*)) di+ 

+ terms independent on i? A / an d analytic in A, (HO) 

where gM (t) = f (p n - fXp^ 1 ([X„] 1 g) W (t\m). 
For scalar or vector-function F (A) let us denote 

A p ( X ) _ F ( X + - F (A) F (A + A m A) - F (A) 
fcm 1 ' A fc A A m A 

Let us denote 



Rn(A)= / (p n (i? A /) [nl (t|m), 5 (n) )di. 
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In view of (I12p . (I68p we have 

i ( r b \ 1/2 

|A fem R n (A)| < (m[A km R x f, A km R x f])i U (p n g^,g^)dt) (111) 

Therefore R n (A) depends analytically on nonreal A in view of analyticity of R\ and so 
analyticity of M (A) is proved in view of (jllOp . 

Let us consider the solution x\(t,F) = 1Z\F (|73p of equation (|67p . Let us prove that 
x\ (t,F) satisfies the condition ([55]) . Let us denote y(t,X,f) = R\f ■ Then in view of Green 
formula (g2|) 



771 [y> 2/] — = o (*> ; a) y (*, Za, m,f),y (t, l\,m, /)) 

yA z 



/3fA (112) 

But the left hand side of (|112jl is < since R\f is generalized resolvent. So 

V/eff: K (Q (*, /a) y m, /) , y (*, *a, m, /3?A < 0. (113) 
But for every % r -valued F (t) G ^w(t l- m) 0^) there exists such vector-function f (t) £ H that 
x A (a, F) = y (a, Z A , m, /), 2; a (6, F) = y (b, l\m, f). So ([55]) is proved in view of (|113p . 

To prove that M(A) is a c.o. of equation (|67[) it remains to show that M(A) = M*(A). 

Let us consider the following operator M (A) G (-^ _L ) : 

M (A) = M (A) , M (A) = M* (A) , 9A > 

This operator is a c.o. of equation (|67p in view of |21| . This c.o. generate by Theorem 13.11 
the operator R (A) (|S2J). 

But i? (A) = i? A) 3A > (A) = R* (A) = R* x = R x , 3?A > => =► V/ € H: 

[X- x (t)] x (M* (A) - M (A)) / (a) VF (s, Z A , m) F (s, h, m)ds =0 

J a m 

=>- V/i G iV x : A A (a, 6) (M (A) - M* (A)) h = =► M (A) = M* (A) . 
Theorem 13.31 is proved. □ 

For generalized resolvents of differential operators a representation of (|82[) type was obtained 
in |35| for the scalar case and in [5] for the case of operator coefficients. For generalized 
resolvents for ([I]), ([2]) with s = 0, ra A [y] = the representation of such a type was obtained in 

Let Xfc, = 1, 2 be finite intervals, X\ C X 2 . Then, in spite of the fact that / (t) G C s (X 2 , H) 
but Xlif (t) & C s (T2, where XXi 1S the characteristic function of X\, one has. 

Corollary 3.2. Let G X\ and the condition (|7ip withX = X2 holds. Let R\ be the generalized 
resolvent of the relation Cq in L 2 m (X) with X = X 2 . Then by Theorems Vd '. H COI there exists c.o. 
M (A) of equation © such that R x f = y 1 (t, A, /) flEE) , t£X = X 2 , f G H (= H (X 2 )). Let us 
define the operator y\ (t, A, /) = R\f, t G X = Xi, f £ H (= H (X±)) by the same formula ([8~T]) 
as operator R\f , but withX = X\ instead ofX = X 2 . TTien this operator is (after closing) the 
generalized resolvent of the relation Co in L^ (X) with X = X\ . 

It is known [H] that ([83]) - (JEgJ implies ©, where E^ £ B (L 2 rn (X)), ^ = E M _ , 

< £ w < E M2 < I, /xi < ^ 2 ; ^-00 = 0. (114) 

Here I is the identity operator in (X). We denote E a p = i [-E^+o + — -Ea+o — E a ]. 
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Theorem 3.4. Let M (A) be the characteristic operator of equation ([5]) (and therefore by |21l 
p. 162] SsPM (A) P/9A >0as9A/0jaridff(/i) = w- lim ± ^ QPM (fi + fe) Ptfyt 6e A/ie 

spectral operator-function that corresponds to PM (A) P. 

Let for equation ([5]) the condition (|7ip urai/i P = L r hold if I is infinite. Let be generalized 
spectral family (|114p corresponding by ([3]) to i/ie resolvent R (A) /rom Theorem 13.11 which is 
constructed with the help of c.o. M (A). T/ien /or any [a, f3] C i/ie equalities 

PE ajj3 f(t) = Pj^[X„(t)] l da(fi)(p(n,f), if f (t) 6 1, 1 is infinite, (n5) 
(*) = /f dCT (/*) V 0, /),*// (<) £ H, I is finite 

are valid in L^ (X), where \X\ (t)] l G B (ri r , Ti) is the first row of the operator solution X\ (t) 
of homogeneous equation (|70h with coefficients ([7|), (jHJ), (|15p . (116p f/ia£ is written in the matrix 
form and such that X\ (0) = I r , 



(/i , /) = J Iz(i x At)]iTm[f}dtiff(t)eH 

1 f x ([X fl (t)] 1 )*W(t,l ll ,m)F(t,l ll ,m)dt, if f{t) G H or f (t) G H, X is finite ' 

(116) 

H G [a,/3]. 

Moreover, if vector-function f (t) satisfy the following conditions 

o o 

PE^f = f, P J^jgdE^f = if f G if, Z «s m/sra'ie 
-Eoo/ = /, / R i\ B dEfjtf = if f £ H, 1 is finite 

then the inverse formulae in L^ (X) 

° ° 
/ (*) = P J B [X M (t)^ do- (/x) </?(//,/) if f (t) e H,X is infinite, 

/ (*) = / B (*)]i ^ (A*) V (m, /),*// (*) G H,X is finite 
and Parcevel's equality 

m[f,g]= / (da(p)<p(ji,f) ,tp(fj, } g)) , (119) 



are valid, where g (t) G H if I is infinite or g (t) G H, if I is finite. 

o 

In general case for f (t) , g (t) G H if X is infinite or f (t) ,g (t) G H if X is finite, the 
inequality of Bessel type 



m [f (t) , g (t)] < / (da (/i) <p (/i, /) , (/i, g)) (120) 
JB 

is valid. 

Let us notice that B = L)k(ak,bk), (aj,bj) (ak,bk) = 0, k ^ j since B is an open set. In 
(fTT8|) P L = Y,k lim P Ia k - In (fTT8|) - (fT20|) we understand L similarly. 

Proof. Let for definiteness r = s = 2n, X is infinite (for another cases the proof becomes 

o 

simpler). Let the vector- functions / (t) , g (t) G H, A = fx + ie, G\ (t, l\,m) be defined by (f26|) 
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with g (t) instead of / (t). In view of the Stieltjes inversion formula, we have 



1 



([yi (t,XJ)- yi {t,\,f)],g) m dn 



lim 



lim 

e^o 2iri 



(y 1 {t,l x ,m,f) ,G(t,l x ,m)) L 2 Ji)-(m (*» l \> m > /) . G (*. *a» m ))z 2 , ,m + 



f M (A) jf X| (t) W (t, m) F (t, l x , m) dt, jf A^ (i) W (t, l\,m) G (t, l x ,m) dt) - 



- M* (A) / A^ (t) W (t, l\,m) F (t, l x ,m) dt, / AT| (t) W (t, l x ,m)G (t, l- x ,m) dt 



dfj, 



da {n) J^X;(t)W(t,l ll )F(t,l^m)dt, J x;(t)W (t,l fi )G l ,(t,l^m)dt\ , (121) 

where the second equality is a corollary of (140 p . the next to last is a corollary of (18 ip and 
the last one follows from the well-known generalization of the Stieltjes inversion formula |35^ 
Proposition (B), p. 803], Lemma, p. 952]. (In the case of finite X we have to substitute in 
(I12ip M (A) by PM(X)P and then when passing to the next to the last equality in (|12ip we 
have to use the remark after the proof of Lemma 12,11 ) But for A £ B 

(122) 



Xl(t)W(t,l- x ,m)F(t,l- x )dt= / ([XxWtfmWdt 



because, in view of ([3 



Vh£H r :{j X{ (t) W (t, l x , m) F (t, l~ x ) dt, h) = 

(W (t, l x ,m)F (t, l x ) , X- x (t) h)dt = ([ ([X-^Y m [/] , h)dt. 

Jx 



Due to (H2U), (fT22]) . (fTTEl) 



{E a ,/3f,g) L 2 



{da (n)^p{nJ),(p{n,g)) 



(123) 



The equality (|119|) and inequality (|120p are the corollaries of (I123p . 

Representing ip(fj,,g) in (|123|) by the second variant of (|116|) . changing in (|123p the order 
of integration and replacing by integral sum and using (I34p we obtain that 



{ E a ,pf,g) L 2 m{x) = ( / [X^^^daii^ip^J) ,g{t)) L 2 m{X ) = 

[X„ (t)l da (/i) <p (ji, f) , g (t)) L 2 AX) (124) 

and (|115p is proved since g(t) G H. Equalities f 1 1 1 8 1) are the corollary of (|115p . (I117p . Theorem 
13.41 is proved. □ 

Let us notice that if X^(X) = L m (X) then Theorem 13.41 is valid without condition (|7ip with 
P = I r if X is infinite. 



28 



VOLODYMYR KHRABUSTOVSKYI 



It is known (see for example |17| or Example 13. 2\ that just in the case n\ [y] = in ([T]), ([2|) 
there is such satisfying flU} - ([85]), (fTT4l) that E^ / I. 

On the other hand if n x [y] = then V/ G D (£ ) #00/ = f in view of pi], PU- 

Let expression n\ in representation (|2|), (|80p have a divergent form with coefficients pj = 

Pj (*, <*) . ?j = Qj (*; A ) > % = 5 i (*> A )- 

We denote m (t) three-diagonal (n + 1) x (n + 1) operator matrix, whose elements under 

main diagonal are equal to ( — . . . , — |g n ) > the elements over the main diagonal are equal to 

(|si, . . . , |s n ) , the elements on the main diagonal are equal to (po, . . . , p n ), where pj, q~j, Sj = 

q- are the coefficients of expressions m. (Here either 2n or 2n + 1 is equal to the order r of 

If order of n\ is less or equal to 2n, we denote n (t, A) the analogues (n + 1) x (n + 1) 

operator matrix with pj,q~j,s~j instead of p~j,q~j,Sj. If order m or order n\ is less than 2n, we 

set the correspondent elements of m (t) or n (t, A) be equal to zero. 

Theorem 3.5. Let in dTJ, ([5]) the order of the expression n\ is less or equal to the order of 
the expression I — Xm (and therefore in view of ft8(J\) the order of I — Am is equal to r; so 
Q (t, l\) = Q (t,l — Am) ). Let y = R\f, f £ H be the generalized resolvent of the relation Co 
and y satisfy equation ([T]). Let y\ = R(X) f,f£Hbe the operator ([FT]) . (|82"j) from Theorem 

roi 

Let the following conditions hold for r > large enough: 
1°. 



lim 



(5RQ (t, l\) (j/i (t, lx,m,f)-y (t, I - Am, m, /)), (yi (t, Z A , m, /)) -y(t,l- Am, m, /)) 



< 



< 0, A = ir (125) 



9n (t,A) <c(t,T)m(t),tEl, A = ir, (126) 
where the scalar function c (t, r) satisfies the following condition: 

sup c (i, r) = o (r) , r — >• +oo. (127) 

T/ien /or generalized spectral family E^ (|114p corresponding by ([3|) to f/ie resolvent R (A) 
([8T|) -(|82p /rom Theorem \3.1\ and for generalized spectral family corresponding to the gener- 
alized resolvent R\ one has E^ = 8^ . 

Let us notice that in view of (|126p the coefficient at the highest derivative in the expression 
I — Xm has inverse from B (H) if t € I, 9A ^ 0. 

Proof. Let / (t) £ H , y± = R (A) f,y = R\f ■ Then z = y\ — y satisfies the following equation 

/ [z] - Xm [z] = n x [ yi ] . (128) 
Applying to the equation (|128[) the Green formula (j42]), one has 



rp rP 
/ 5$(n\{y 1 ,z})dt+ / m{z,z} 

J a J a 



dt= m{Q(t,h)z,z) 



2 9A 
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m \z, z] < 



< 



where z = z(t, I — Am,, n\, y\) = y\ (t, l\,m, f)—y(t,l — Am, m, /) in view of ([28]) and Lemma 
1.21 Hence for r > large enough 

J ^ ( n x [yi,z])dt/T < 

(n (t,A) col {yi,^,..., 2/j n) } , col [z, z' , . . . , | dt It, A = ir (129) 

in view of (I125p . But due to the inequality of the Cauchy type for dissipative operators [35} p. 
199] and (jl 261) . fjl2T[) : subintegral function in (I129p is less or equal to (m {z, z}) 1 ^ 2 (m {2/1, 2/ij) 1 ^ 2 c 
with A = it, t — > +00. Therefore \\z\\ m < o(l/r) ||/|| m since \\R\\\ < 1/|9A|. Hence 

\\R(X)-Rx\\<o(1/t), X = ir, r^+00 

To complete the proof of the theorem it remains to prove the following 

dE^ 

Lemma 3.4. Let R^ (A) = J Kl — k = 1,2, where E^ are the generalized spectral families 



the type (flTlj) in Hilbert space H. If ||J?i (A) - R 2 (A)|| < o(l/r) , A = ir, t 

Proof. Let / GH o (ji) = E l) /, /). One has 

\([R 1 (X)-R 2 (X)]f,f)\ = 



+00, then 



da (fx) + 



/j,da (fi) 



A V 



A 



+ 



Xdo (ju) 



Therefore 



(f<7 (/i) + 



/idu (fi) 



A V 



A 



+ 



fli\A A 1 - A 
A(i<T (/i) 



<o(l/r) 



X = ir, T — > +00 



A 



< o (1) , A = ir, r — >• +00. 



(130) 



For an arbitrarily small e > we choose such finite interval A (e) that for any finite interval 
ADA(e): J Rl 



\A ^t— A 



< 



A 



Vr > JV : 



< 



- ir. But for any finite interval A D A (e) 3iV = iV (A) : 

, A = ir. Therefore Ve > 0, A D A (e) : I J" A dcr < e in view of 

PDJ). Hence V/ eH: /) = (£&/, /) =► = 

Lemma 13.31 and Theorem 13.51 are proved. □ 

□ 

Corollary 3.3. Let the conditions of Theorems \3.4\ Iff- 51 hold. Then for generalized spectral 
family from Theorem V/ (t) 6 D (£0) : ^00/ = /• 

o 

Remark 3.1. If (X) = (X), then it is sufficient to verify condition (|125p in Theorem 13.51 
for f€H. 

Proposition 3.2. Let the order of expression n\ be less or equal to the order of expression 
I — Am and the coefficient of I — Am at the highest derivative has the inverse from B (TL) for 
t £ X, X £ B (I — Am), where B (I — Am) is an analogue of the set B = B Let interval I 
be finite and for equation ([I]), ([2]) with n\ [y] = condition (|7ip holds with P = I r . Then for 
equation (|2|) this condition also holds with P = L r and resolvents y = R\f, 2/1 = R{X) f 
from Theorem \3.5\ satisfies condition (|125p for SA ^ if they are the solutions of boundary 
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value problems for equations ([T]) ; ([2]), (n\ [y] = 0) and ([I]), ([2|) with boundary conditions from 
Theorem \3.S\ with the same operators M\, M\. 

Proof. In view of Theorem 13.21 it is sufficient to prove only proposition about condition (17ip . 
Let for definiteness order I = order m = order n\ = 2n. 

Let for equation (pQ), ([2]), (n\ [y] = 0) condition (|7ip with P = I r hold, but for equation ([TJ, 
(|2|) that is not true. Then in view of |21| the solutions y k (i) of equation ([1]), ([2]) with A = i 
exist for which 



(m + Qm){y k ,y k }dt^0, y k (0, k, m, 0) = f k , \\f k \\ = 1, 



:i31) 



where zQnj = rij in view of (I80p . Hence in view of (I34p 

,/3 



(Wi(t,l + im,ni)Y k (t,l + im,ni),Y k (t,l + im,ni))dt= f n\ {y k , y k } dt ->■ 0. (132) 

./a 



On the other hand 



(t) / fc = ^ (i) f k + X (t) / XT 1 ( a ) J~ l W (s, I + im, m) Y k (s, / + im, n t ) ds. (133) 

J o 

in view of Theorem 11.11 and the fact that y k (t,l — im,rii,y k ) = y k (t,li,m,0), where X\ (t) is 
an analogue of X\ (t) for the case n\ [y] = 0. 
Comparing (I132p . (1133P we see that 



Xi^h-Xi^h 

uniformly in t G [a, /3] . 

In view of (j 1 3 1 [) subsequence exist such that 



(134) 



m {yk q ,Vk q } ^'0, ni{2/ fc5 ,y fc J a 4'0. (135) 
Due to second proposition (|135p and the arguments as in the proof of Proposition 13.11 one has 

(136) 

Let us denote y k (t) = Xi (t) f k . In view of Theorem 11.11 and (j!34p 



Vk] (* N)"^' j = n,...,2n. 



0) 

9 



(ft. (t) - ift. (*)) »2f(t)-C(t) 



->■ 0, j = l,...,n- 1, 



(137) 



-(q n (t) - iq n (t)) [y£ _1) (t) - t q ~ l \t)\ - v [ k q ] (t \m 

y [ k q ] (t\h) ~ y [ kq ] (t\l ~ ^ (138) 

uniformly in t G [a, /?]. Comparing (|135p . (|136p . f|138j) and using (p n (i) — ip n (t)) 1 G B {%) 
we have 

(139) 



0. 



{Pn(t)yr t >(t), 
In view of (TTBTD . (HMD, (gMD 

m{y kq , yt q } ^ 0, j4 a (0, i - im, m, 0) = / fc? , 

that contradicts to the condition (|7ip with P = I r for equation ([1]), ([2|) with n,\ [y] = 0. 
Proposition 13.21 is proved. □ 
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In the next theorem X = R 1 and condition (I7T1) hold with P = I r both on the negative 
semi-axis R- (i.e. as X = and on the positive semi-axis R + (i.e. as X = R+). 

Theorem 3.6. Let X = R 1 , i/ie coefficient of the expression l\ (j2j) 6e periodic on each of the 
semi-axes R- and R + with periods T_ > and T + > correspondingly. Then the spectrums 
of the monodromy operators X\ (±T±) (X\ (t) is from Theorem \3. 1\) do not intersect the unit 
circle as 9 A ^ 0, the c.o. M (A) of the equation (JSJ) is unique and equal to 

M (A) = (v (A) - M (iGT 1 (3A / 0) , (140) 

where the projection V (A) = P+ (A) (P+ (A) + P_ (A)) -1 , P± (A) are Riesz projections of the 
monodromy operators X\ (±T±) i/iai correspond to their spectrums lying inside the unit circle, 
(P+ (A) + P_ (A))" 1 G 5 (rT) as 9A / 0. 

y4/so let dim% < oo, a finite interval A C Z3. Then in Theorem \3.4\ da (p) = da ac (li) + 
dad (m) j A* e A- i/ere <r ac (/i) G AC (A) and, /or /i G A, 

<c (aO = ^g- 1 (q*_ ( M ) gq_ ( M ) - q; ( M ) gq+ ( m )) g- 1 , 

where the projections Q± (li) = q± (/x) (P+ (/x) + P_ , a-t (/x) are Riesz projections of the 

monodromy matrixes X^ (±T±) corresponding to the multiplicators belonging to the unit circle 
and such that they are shifted inside the unit circle as /x is shifted to the upper half plane, 
P± (m) = P± (A* + *0) I °d (A*) * s a s ^ e ] 5 function. 

Let us notice that the sets on which g± (li) , P± (/x) , (P + (/x) + P_ (/x))~ are not infinitely 
differentiable do not have finite limit points G B as well as the set of points of increase of 

Proof. The proof of Theorem 13.61 is similar to that on in the case n\ [y] = |23| . □ 

The following examples demonstrate effects that are the results of appearance in l\ ([2|) of 
perturbation n\ depending nonlinearly on A. 

In Examples 13.31 [3~4l nonlinear in A perturbation does not change the type of the spectrum. 

o 

In this examples dim U = 1, m[y] = -y" +y. L 2 m (X) = L 2 m (X) = W^ 2 {M}) . In Example [33] 
such perturbation implies an appearance of spectral gap with "eigenvalue" in this gap. 

Example 3.3. Let 

h [y] = iy' - A {-y" +y)-(~y), h> 0. 

Here B = C \ 0, Eq = E+q, spectral matrix a (li) G ACi oc , 

a' (li) = 0, as |/x| > y/h + 1/4. 

In Example 13.31 nonlinear in A perturbation change edges of spectral band. 

Example 3.4. Let 




h > 0. 
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Here B 



C\{0}, h 7^0 
C, h = 



Eq = E + q , spectral matrix a (/x) £ AC\ C 



a'(/x) 



j_ 

2tt 



A+vTJ 



2tt 






't 







1 

2 





1 





o 









V 



o o 



■i \ 





-i { 






as 



as /x* < /x < V~h, 



where D = A 2 —4(7, g = h/X — A, /i* = /x* (/i) - nonnegative root of equation D = 0. cr' (/x) = 0, 
as /i ^ — |J[/i*,oo). 

In Example 13.41 nonlinear in A perturbation implies an appearance of additional spectral 
band [— h, 0], variation of edge of semi in finite spectral band and appearance of interval 

fi* , y/h) of fourfold spectrum. 



Example 3.5. Let dim% = 2 
^a [y] = 
'<C\{0}, h^O 



-1 

1 



y - Ay 



-h/X 




y, h>0. 



Here B 



h = 



spectral matrix a (/x) = a ac (n)+a d (//), a ac {^) G AQ oc , cr^, (/x) / 



0, as > \J~h, a' ac (/x) = 0, as |/x| < step-function cr^ (//) has only one jump 




y/hll 



in point /i = (inside of spectral gap). In this point 

/ 

(E +0 -E )f = Q 



^l*- s l/(s)d S) /(t) e L 2 (IR 1 ) . 



Let us notice that in view of Floquet theorem conditions of Theorem l3.5l ( (|125p with account 
of Remark 13. ip hold for all Examples I3.3H3.51 
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